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AIR FORCE SPACE COMMAND ACTION OFFICER GUIDE TO 
RELIABILITY AND MAINTAINABILITY 

PREFACE 


The Secretary and Chief of Staff of the Air Force stated 
the Air Force commitment to reliability and maintainability 
(R&M) in their joint memorandum of September 1984 and renewed 
the commitment in July 1986. An Air Force R&M 2000 Action 
Plan was developed to provide general policy and guidance to 
institutionalize R&M in the way we do business - both now and 
in the future. 

In response, Air Force Space Command/LKYY outlined 
Command goals in the Reliability and Maintainability Program 
Plan which provides approaches to improve R&M performance and 
achieve AF R&M 2000 goals. This R&M Action Officer Guide is 
an integral part of LKYY's efforts to expand the R&M Program 
Plan so that AF Space Command action officers can understand 
R&M parameters and how they impact system capability and 
performance. This guide illustrates how R&M principles 
should be applied to new acquisition programs as well as to 
fielded systems. 

I fully endorse this Guide and expect all AF Space 
Command action officers who are involved in the acquisition 
process to not only familiarize themselves with its contents 
but, more importantly, to utilize this information to ensure 
we get low cost, high performance space systems. 


DONALD J. KUTYNA 
Lieutenant General, USAF 
Commander 
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1.0 INTRODUCTION 
1.1 BACKGROUND 


Although R&M had long been recognized as important 
considerations in fielding major weapon systems, the emphasis on 
R&M in an acquisition program usually took a back seat to cost, 
schedule and performance, especially when trade-offs were 
required to offset budgetary constraints. These cost cutting 
practices reduced up front development/procurement costs but 
resulted in increased long term operations and support costs. To 
combat the mounting cost of supporting our space and ground 
systems, the Secretary and Chief of Staff of the Air Force have 
promulgated vigorous Air Force commitment to R&M: 


"For too long, the reliability and maintainability of our 
weapon systems have been secondary considerations in the 
acquisition process. It is time to change this practice and 
make R&M primary considerations." 

"We must emphasize R&M throughout the acquisition process— 
from requirements definition, throughout concept development, 
design, production and acceptance. Everyone must ensure R&M 
requirements are met through every step of the process. R&M 
must be coequal with cost, schedule and performance as we 
bring a system into the Air Force inventory." 

(General Charles Gabriel, Chief of Staff, USAF, 
and Mr. Verne Orr, SAF, 17 Sep 1984) 


1.2 PURPOSE OF GUIDE 


This guide is designed to help action officers responsible 
for managing acquisition programs better understand the 
interrelationship between R&M performance and operational 
capability. It won't turn you into an R&M engineer. 

This guide is designed to be used with the Command 
Management Guide developed by XPT and the Reliability and 
Maintainability Terms for Space. Space Surveillance, and Missile 
Warning Systems technical report published by LKY. 

The XPT guide is a comprehensive document that provides a 
single source of reference for the staff officer. It contains an 
overview of the acquisition process and outlines the command 
manager's responsibilities. For more information, contact HQ 
AFSPACECOM/XPT, Peterson AFB, CO 80914-5001 or call AV 692-5668. 
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The LKY technical report details both mission and logistics 
Support R&M parameters. The terms and definitions contained in 
the report are approved for use in SOtfS and SORDS. To obtain a 
copy of this report, contact HQ AFSPACECOM/LKYY, Peterson AFB, CO 
80914-5661 or Call AV 692-3286/5898. 


1.3 OVERVIEW 

The basic guide discusses the role of R&M in requirements 
determination and system acquisition process. Appendix A is a 
primer of basic R&M concepts and computations. Appendix B is a 
checklist designed to evaluate R&M content in System Operational 
Requirements Documents (SORDs). Contract data items applicable 
to R&M are listed in Appendix G. 




2.0 R&M IN THE SYSTEM'S LIFE CYCLE 


2.1 OVERVIEW OF PHASES 

System acquisition programs normally go through several 
phases, each with particular milestones, as they progress from 
the identification of a need to final operational deployment. 
Figure 2-1 shows the acquisition process and the major 
events/documents that usually occur in each phase. Phases may be 
combined or omitted depending on circumstances; however, each 
phase is designed to develop a level of confidence in the 
solutions offered and to reduce the risks of entering the next 
phase. R&M is a key factor in the decision process as the 
program progresses through each phase. 


2.2 MISSION AREA ANALYSIS PHASE 

The acquisition process begins with a mission area analysis 
identifying mission requirements and assessing the Command's 
capability to fulfill each requirement. Statements of 
Operational Need (SONs) define new requirements that cannot be 
met through changes in tactics, strategy, doctrine, or training. 
The SON also offers a possible solution involving either new 
equipment or upgrades to an existing system. Section 3 describes 
placing R&M requirements on SONs. Once the SON is validated by 
the Air Staff, it forms the basis for writing the Program 
Management Directive (PMD). The System Program Office (SPO) 
selected for the project then develops a Program Management Plan 
(PMP) that describes the tasks necessary to develop a system that 
fulfills PMD requirements. The operational command is also 
tasked to develop a System Operations Requirements Document 
(SORD) which amplifies and refines the operational and support 
needs specified in the SON. R&M considerations for these 
documents are discussed in Section 3. 


2.3 CONCEPT EXPLORATION/DEFINITION fCE/D^ PHASE 

In the CE/D phase, the implementing command (normally AFSC 
for a major new weapon system) may have several companies in 
competition for the award winning design. Operations and support 
planning are integral activities during this phase. Contractual 
R&M requirements are developed to the same extent as are other 
performance parameters. As system operating and support concepts 
are further developed during the acquisition process, operational 
R&M needs and the corresponding contractual R&M requirements are 
challenged and refined. The objective of this refinement process 
is to have R&M needs and contractual requirements that are 
validated, consistent, achievable and acceptable by the 
operating, implementing, and supporting commands prior to 
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developing the Full Scale Development Request for Proposal (FSD 
RFP). Section 5 explains R&M tasks in the contractual process. 

By the end of the CE/D phase, decisions affecting 70% of the 
Life Cycle Costs (LCC) have already been made, and decisions 
affecting 85% of the LCC are made prior to Full Scale Development 
(FSD), as shown in Figure 2-2. These decisions affect system 
supportability, R&M characteristics, and LCC. For maximum 
benefit, R&M requirements must be clearly stated in early 
acquisition documents including the System Operational 
Requirements Document (SORD), the R&M Management Plan (RMMP), and 
the Test and Evaluation Master Plan (TEMP). These documents are 
discussed in Sections 3 and 6. 


LIFE-CYCLE COSTING IN SYSTEM ACQUISITION 




SYSTEM LIFE CYCLE 


YEARS 


Figure 2-2 


CONCEPT DEMONSTRATION/VALIDATION (CD/Vi PHASE 


When the exploration of alternative system concepts has been 
completed to the point where the selected alternative warrants 
system demonstration, the program enters the CD/V Phase. The 
purpose of this phase is to reduce technical risk and economic 
uncertainty through a more detailed definition of the system. 

The System Program Office (SPO) works closely with the 
contractor(s) to further define the system and will frequently 
task them to build prototypes and/or accomplish desktop analysis 
to provide necessary detail. These details are included as 
updates to the documents first published in the Concept 
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Exploration/Definition phase. Therefore, much of the R&M work in 
this phase of the acquisition is a continuance of work done 
previously. R&M requirements are allocated down to the 
subsystems and/or equipment items within the system. Certain 
meetings (i.e.. System Requirements Review (SRR) and System 
Design Review (SDR)) are held to review the system requirements 
and detailed design. R&M requirements for these reviews are 
described in Section 5. 


2.5 FULL SCALE DEVELOPMENT (FSD^ PHASE 

Upon final selection of a design concept, the system is 
ready to enter the FSD Phase. The implementing command will 
reaffirm the operational need for the system, adequacy of 
evaluation of the alternative approaches, adequacy of the test 
and evaluation approach, sufficiency of cost and schedule 
estimates, and consistency of the acquisition strategy and 
contractual plan with program characteristics, requirements and 
risk. Once this review, called the Secretary's Program Review 
(SPR) is completed, the implementing command will enter FSD. 
Contract requirements for R&M are the same as those in the 
earlier phases (see Section 4). Once the contract is awarded, 
two major reviews, the Preliminary Design Review (PDR) and 
Critical Design Review (CDR), are conducted to check the 
contractor's progress and ensure the technical adequacy of the 
design approach. During this phase, the action officer must 
review the test planning documentation to evaluate the DT&E and 
IOT&E efforts against the operational requirements. Once the 
test has been conducted, there are reviews of the test results to 
determine how well the actual system meets operational and 
support requirements. R&M is a determining factor in how well 
the system passed OT&E, as described in Section 7. Finally, 
there is another SPR at the end of FSD to make the final decision 
to accept the prototype as a one of a kind or to mass produce the 
system. R&M issues for this SPR are identical to those of the 
previous SPR, described in Section 6. 


2.6 PRODUCTION/DEPLOYMENT (P/Dl PHASE 

Once the system design has been fully developed, approval 
can be given to enter the P/D phase. For a major program 
involving large production quantities, producibility and 
reliability growth are examined. Large production runs don't 
normally occur for the typical Space Command acquisition; the 
prototype system frequently becomes the operational system. 
Sometimes an evolutionary acquisition strategy is used. Under 
this strategy, system capabilities are acquired in blocks. Each 
block is sequentially acquired and employed and becomes the base 
for the next block. This strategy is used primarily on systems 
which push the state of the art or on which future requirements 
cannot be accurately forecast. Tracking R&M requirements across 
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each evolutionary acquisition block is critical to achieving 
operational capability. 


2.7 OPERATIONS SUPPORT PHASE 

The OS phase begins with the first operational deployment. 
The system's operational performance is tracked and problems 
identified. R&M parameters affecting mission performance or 
supportability are closely monitored. Equipment replacement or 
modification may be required to maintain system level R&M 
performance. The importance of maintenance data collection 
during this phase is addressed in section 9. 


2.7.1 Initial Operational Capability (IOC) . At the beginning of 
IOC, the using command,(in this case AFSPACECOM), takes 
operational responsibility IAW AFR 800-19. Normally some time 
later, management responsibility for the system transfers to AFLC 
as part of Program Management Responsibility Transfer (PMRT), IAW 
AFR 800-4. R&M tasks in this phase are normally limited to 
Follow on OT&E (FOT&E) (described in Section 8), and the 
establishment of a Maintenance Data Collection (MDC) System to 
track actual R&M performance. 


2.7.2 Full Operational Capability fFOCl . Once the system is 
fielded, efforts are made to improve system effectiveness and 
safety. The acquiring agency continues to correct operational 
R&M deficiencies caused by material, software, or firmware design 
and quality. The program manager corrects operational R&M 
deficiencies within his/her responsibility and assures that 
operational R&M needs are met. The operating and support 
commands correct deficiencies that are the result of inadequate 
operating and support concepts. After PMRT, the operational R&M 
performance is monitored and reported through a MDC system. 
Analysis is performed to assess operational R&M performance, 
identify deficiencies, and help identify corrective actions. 

This becomes the basis for a new SON, thus reinitiating the 
process. 


2-5 



3.0 R&M IN PROGRAM DOCUMENTS 

The Statement of Need (SON) documents the operational 
command's requirements. Once the SON has been validated and 
funding acquired for the program, the program enters the concept 
exploration phase. At this point in the acquisition, three 
documents are important: the PMD, PMP and SORD. 

3.1 STATEMENT OF OPERATIONAL NEED fSON) 

A SON defines an operational need and documents official 
validation of the need. A SON is an AF-related document, other 
documents that describe a need are the Joint Statement of 
Requirements (JSOR), Mission Need Statement, Required Operational 
Capability (ROC) or Communications-Computer System Requirements 
Document (CSRD). While the documents differ in form and content, 
putting R&M requirements into each of them follows a similar 
logic flow. 

3.1.1 Defining Requirements . The first step in adequately 
preparing a SON with the proper R&M terms is to understand that 
SONs document the need for a new or improved capability, identify 
operational deficiencies, and define constraints on acceptable 
solutions. During the SON development, R&M parameters are 
defined at the system level. Stating top level R&M needs early 
in the acquisition cycle helps determine the best design solution 
by ensuring R&M considerations are an inherent part of the system 
design process. 

Inherent in the mission need are certain top level 
readiness requirements. These readiness requirements should 
relate to the peacetime and wartime scenarios envisioned. For 
example, if you are writing a SON for a ground communications 
system, general wartime requirements from the War and 
Mobilization Plan-3 (WMP-3) might specify the equipment be 
operated 24 hours a day/7 days a week. If the equipment is on a 
transportable platform, it may be required to operate without 
resupply for a specified minimum period of time. 

3.1.2 Requirements Allocations . Once top-level operational 
requirements are established, the R&M values should flow down 
from and support these requirements. Most Space Command systems 
are low-density (normally one per geographic location and less 
than 20 locations worldwide) or one-of-a-kind space and attack 
warning systems. These systems often have to meet extremely high 
R&M effectiveness criteria. To develop or evaluate R&M 
requirements, the operational need must be completely understood. 
Factors including the system's mission, its operations concept 
and maintenance concept must be considered. 
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As a minimum, the following R&M parameters must be 
determined and included in Section IV A of the SON: 

— Mission Reliability 
— Mean Mission Duration 
— Mean or Maximum Restoral Time 
— Availability/Dependability 

For further information on the use/application of these 
terms, consult AFSPACECOM/LKY Technical Report 88-1, standard 
R&M Terms for Space. Space Surveillance, and Missile Warning 
Systems, dated 20 April 1988. 


3.1.3 Quantifying Requirements Once the appropriate terms have 
been selected, the next decision is how much R&M to ask for. 

This decision should be based on mission requirements, 
technology, and comparable current systems. This task can be 
difficult due to system complexity and incomplete historical data 
on predecessor system outages. Also, it's not always possible to 
find a "like system" to use as a basis for comparison. Many of 
our new systems provide capabilities not previously available. 

All possible sources of information should be tapped. 
Dialogue with contractors and engineers at Space Division and 
Electronic Systems Division, as well as the AF Acquisition 
Logistics Center (AFALC) and the AF Coordinating Office on 
Logistics Research (AFCOLR) can provide valuable information. 
Additionally, AFSPACECOM/LKYY personnel will assist you in this 
effort. 

In AF Space Command, all SONS must process through LKYY to 
ensure adequate R&M parameters are inserted in the document. 
Figure 3-1 shows the AFSPACECOM SON validation process which 
ensures R&M is adequately addressed. 

3.2 PROGRAM MANAGEMENT DIRECTIVE (PMD^ 

The PMD is the master program document for any 800-series 
acquisition. It defines the responsibilities and funding 
profile for the program and is the key decision-making tool to 
coordinate the efforts of the using/developing/supporting 
commands. As far as R&M are concerned, the PMD should contain a 
section under "Implementing Command Responsibilities" that shows 
what tasks are going to be performed as part of the program. 

Those tasks include conducting an R&M program per AFR 800-18, 
publishing an R&M Management Plan (see Section 7.4 for details) 
and integrating R&M into the logistics program. R&M tasks must 
be inserted in the first version of the PMD. Planning up front 
for an R&M program is significantly easier than trying to acquire 
it later because the common "bring money" arguments with the SPO 
can occur if the requirement for a sound R&M program is not 
specified up front in the PMD. 
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3.3 PROGRAM MANAGEMENT PLAN (PMP) 

The PMP is the SPO's response to the PMD. Instructions on 
how to prepare it are contained in AFR 800-2. All tasks required 
by the PMD should show up consistently in the PMP. R&M should, 
as a minimum, be included in section 4 (System Engineering and 
Configuration Management), section 5 (Test and Evaluation), and 
section 9 (Logistics). If the PMD has clearly defined the R&M 
program's requirements, then the SPO should show how it will 
complete all required actions. Inadequate discussion of the 
R&M program in the PMP is one of the first indications of future 
trouble in getting the SPO to sign up to R&M. 


3.4 SYSTEM OPERATIONAL REQUIREMENTS DOCUMENT (SORD) 

A SORD is submitted by the operating command for each 
funded program as tasked in the PMD. The SORD is the 
requirements and planning document prepared to address 
operational and support needs. It amplifies and refines the SON. 
AFR 57-1 specifies the SORD format and directs that the SORD will 
quantify the following R&M parameters: 

A) mission reliability and maintainability 

B) logistics reliability and maintainability 
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C) operational availability and dependability 

These values are described in Section IV.A.1 of the SORD. 
Section IV.A.l.a specifies the required system performance 
parameters such as capacity, mission duration, reaction time, 
etc. Section IV.A.l.b provides the mission reliability 
and maintainability parameters that the system must exhibit (see 
the R&M Primer and AFSPACECOM Technical Report for Standard R&M 
Terminology for further details). Section IV.A.l.c covers the 
logistics R&M (expressed as Mean Time Between Maintenance (MTBM) 
and Mean Time to Repair (MTTR)) for the system. Finally, Section 
IV.A.l.d describes the readiness measures in terms of Ao or 
operational dependability (Do). These four sets of parameters 
should be interrelated; i.e., the operational parameters should 
form the basis for the R&M parameters that follow. AFSPACECOM 
SORDs must be coordinated with LKYY who uses the SORD checklist 
in Appendix B to assess whether R&M issues have been adequately 
addressed. 


3-4 



4.0 R&M IN THE CONTRACTUAL PROCESS 

The SON and SORD formalize Space Command's operational 
requirements. In these documents, needed capabilities are 
described in terms of mission requirements, operational 
objectives, employment, and support and maintenance concepts. 

The SPO then takes the operational R&M parameters stated in the 
SON and SORD and translates them into contractual teotis such as 
Mean Time Between Failures (MTBF), Mean Time to Repair (MTTR), 
etc., to meet the SON/SORD requirements. Next, the SPO solicits 
private industry and public agencies for proposed solutions to 
this need in the Request for Proposal (RFP). The RFP includes a 
model contract with a SOW, System Specifications, and Contractor 
Data Requirement List (CDRL). The AFSPACECOM action officer must 
ensure that contractual terms in the specification and R&M tasks 
outlined in the SOW will meet SON requirements. The R&M 
parameters established in the SON/SORD and translated into 
contractual terms by the SPO hold the contractor liable. 


4.1 SPECIFICATIONS 

R&M values are normally contained in Sections 3.2.3 through 
3.2.5 of the specification. Section 3.2.3 should have 
reliability, which will give the required MTBF or R(t) and 
mission duration. Both MTBF and R(t) should be specified. 

Section 3.2.4 gives the maintainability specifications, usually 
in terms of MTTR. Sometimes the SPO uses the phrase: "consistent 
with the reliability and availability values," but it is better 
to specify the requirement. As shown in the R&M Primer (Appendix 
A), MTTR is a contractual term that does not include logistics or 
administrative delay times. Therefore, the MTTR value must be 
less than the operational value given in the SORD. Section 3.2.5 
lists the availability requirement which should be consistent 
with the R&M specifications above. The availability formula 
given in LKY/TR88-1 should balance when combining the RM&A values 
in these three sections. 


4.2 STATEMENTS OF WORK (SOWsl 

A SOW is the part of a contract detailing tasks the 
contractor must perform. R&M tasks in the normal acquisition 
contract include conducting an R&M program IAW MIL-STDs 471 and 
781. References are also included for the development of various 
R&M related data. The details of the data requirements are 
included on the Contract Data Requirements List (CDRL). 


4.3 CONTRACT DATA REQUIREMENTS LIST fCDRL^ 

The CDRL contains the requirements for data to be delivered 
to the government under the terms of the contract. The CDRL 
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contains or references the required format, delivery 
instructions, number and types of copies required, approval 
codes, frequency of updates and required delivery dates. The 
CDRL references a Data Item Description (DID) which describes a 
standard format for commonly requested data. These DIDs can be 
tailored to meet program specific requirements on the CDRL. 
Appendix C contains commonly used R&M DIDs. AFSPACECOM/LKYY will 
assist the action officer in selecting and tailoring these DIDs 
and in preparing the required CDRL information. 


4.4 SOURCE SELECTION EVALUATION 

Source selection is the process of choosing the 
contractor(s) who will design or build the system. Evaluation of 
the contractor's approach to R&M is a critical activity during 
source selection. The contractor must show his or her 
understanding of the R&M tasks, ability to perform those tasks; 
and a sound plan to integrate the R&M tasks into the entire 
system engineering and logistics support effort. Ideally, a 
logistician who can analyze both the logistics and R&M parts of 
the contractor's proposals will be a member of the Source 
Selection Team. 


5.0 R&M IN DESIGN REVIEWS 


5.1 SYSTEM REQUIREMENTS REVIEW CSRR) 

The SRR is conducted to evaluate the contractor's 
responsiveness to the SOW and to ensure the contractor's 
interpretation of the system requirements is correct. At this 
time, it is the Action Officer's responsibility to ensure the R&M 
values originally placed in the SON/SORD were accurately 
translated by the contractor into the Proposal and that all 
elements of the R&M program are in place. This task should 
entail matching the Proposal to the SOW. 


5.2 SYSTEM DESIGN REVIEW (SDR) 

At the SDR, the contractor's proposed approach to meeting 
system operational requirements is reviewed. Normally this 
approach is documented in a draft "A” level specification. The 
R&M emphasis at SDR is the proper and consistent establishment of 
system level R&M design requirements. The R&M requirements 
documented in the "A" level specifications are the inherent 
capabilities the system must possess, given a stated system 
operations and maintenance concept. "A” level specification 
requirements are derived from the SON/SORD requirements by taking 
into account the impact of factors external to the system 
hardware and software. Often the inherent design requirements 
(i.e., "A" specifications) are higher than the SON/SORD 
requirements. The combined effects of inherent and external R&M 
factors are evaluated at SDR to determine if the contractor's 
proposed design is capable of meeting operational requirements. 


5.3 PRELIMINARY DESIGN REVIEW (PDR) 

The PDR is an important check on the contractor's progress 
and consistency and the technical adequacy of the design and 
test approach. It is held during FSD to evaluate the 
contractor's preliminary design effort. The contractor's 
functional allocation of system level requirements to individual 
configuration items of hardware and software is assessed to 
determine existing and potential problems related to system 
capability, equipment engineering and manufacture, and logistics 
supportability. The R&M emphasis at PDR is review of the 
contractor's proposed R&M allocations and predictions. Upon 
authentication of the "B" level specification following PDR, 
these allocated R&M requirements become design requirements 
and are included in the system's allocated baseline. 
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5.4 CRITICAL DESIGN REVIEW (CDR1 


The next major system design milestone after the PDR is the 
CDR. Once the contractor has successfully completed the PDR, the 
detailed design effort begins. Hardware and software are chosen 
or designed to meet the allocated functions. At CDR the 
contractor's detailed design effort is evaluated against both 
system and functional specifications. The SPO acts as the single 
interface between other government offices and the contractor. 

The CDR is held to evaluate the final detailed design prior 
to production. It will verify the functionality, producibility, 
and supportability of the basic design and attempt to catch 
oversights prior to the start of production. Heavy investments 
are under way at this point, so the design must be frozen and all 
items given to final reevaluation in order to minimize risk. 

The R&M emphasis at CDR is the allocation of configuration 
item level R&M requirements down to individual hardware and 
software components. Assessment of maintainability parameters 
against the maintenance concept and support constraints is also 
accomplished. Upon authentication of the "C" level 
specifications, the system's product baseline is established. 



6.0 R&M IN THE DEVELOPMENT PROCESS 


6.1 PROGRAM MANAGEMENT REVIEWS fPMRs^ 

PMRs are held periodically on some programs to allow the 
using, developing and supporting commands to review the progress 
of the system acquisition effort, define problems and look for 
solutions. The status of the logistics section of the program 
(including the R&M program) should be reviewed. 


6.2 SECRETARY'S PROGRAM REVIEWS fSPRs^ 

For designated programs, the SPO is regularly required to 
report the status of the program to the Secretary of the Air 
Force and the Air Staff. Included is a section on the status of 
the R&M Program. Figures 6-1 and 6-2 give examples of the slide 
format used in the SPR. This material should be reviewed and 
coordinated through HQ AFSPACECOM/LKYY before going to the Air 
Staff. 


6.3 LOGISTICS SUPPORT ANALYSIS (LSA) 

LSA is a subset of the system engineering process in 
which support factors influencing system design and support 
requirements are determined. R&M parameters play a major role in 
LSA and are tailored for each program. LSA tasks are contained 
in MIL-STD-1388-1A; MIL-STD-1388-2A describes a standard LSA 
Record (LSAR) system. LSAR data sheets contain key R&M 
parameters at the system as well as the component level. System 
level R&M requirements are documented on the "A" sheet while the 
B and B1 sheets contain key component level R&M data elements. 

The LSAR is designed to serve as the single point data base 
for logistics related design information. Reliability 
predictions; failure modes, effects, and criticality analyses; 
and maintenance manpower and equipment requirement determinations 
are made from and documented in the LSAR. Using the LSAR as the 
single point data base insures that analyses based on R&M 
parameters (e.g.. Life Cycle Costing (LCC), Repair Level Analysis 
(RLA), reliability predictions, spares computations, and tasks 
analyses) utilize the same values. 

The A sheet is completed during the CE/D phase as a result 
of LSA subtask 205.2.3 (specification requirements) of 
MIL—STD—1388—1A. It must be available prior to, or concurrent 
with initiation of LSA subtask 301 (functional requirements) in 
the CD/V phase. The action officer should obtain a copy of the A 
sheet and ensure that the logistics parameters listed will 
satisfy the SON R&M requirements. 
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Since R&M and the ILS elements establish the criteria for 
the entire LSA process, it is important that the A sheet gets 
completed early on to allow the up-front logistics decisions 
necessary to field a reliable and maintainable system. The SPO 
will include information on the application of the LSA process 
in the Integrated Logistics Support Plan (ILSP). 


6.4 SYSTEM PROGRAM OFFICE R&M DOCUMENTATION 

The SPO is responsible for writing two important documents 
related to R&M during system development: the R&M Management 
Plan (RMMP) and the ILSP. 


6.4.1. R&M Management Plan fRMMPl . The RMMP explains the 
program management approach/objectives and describes its R&M 
program for the acquisition. The RMMP must relate to HQ USAF and 
AFSPACECOM R&M plans/goals and form the basis for R&M program 
reviews. The action officer should: 

(a) review the RMMP to verify that R&M has been 
integrated into the entire acquisition/support 
process. 

(b) follow up to ensure the plan is being actively 
executed. 

Additionally, maintainability demonstrations (M demos) or 
assessments are conducted to evaluate the testability, access, 
and types of tools needed for maintenance. 


6.4.2. Integrated Logistics Support Plan (ILSP) . The ILSP is 
the key document for the overall logistics support of the new 
system. The ILSP is an expansion of section 9 of the Program 
Management Plan discussed in para 3.3. It's prepared IAW AFR 
800-8, "Integrated Logistics Support Program." The ILSP covers 
logistics activities throughout the system's life cycle by 
developing an integrated series of individual plans covering the 
different elements of logistics support. One of these elements, 
Design Interface, includes the R&M program. The ILSP should 
adequately document the status of the R&M program and show how 
R&M are being used/integrated into the total ILS effort. 


6.5 INTEGRATED LOGISTICS SUPPORT MANAGEMENT TEAM MEETINGS 

Integrated Logistics Support Management Team (ILSMT) 

Meetings are conducted quarterly. Normally, the ILSMT Meeting 
is chaired by the Deputy Program Manager for Logistics (DPML) and 
is normally attended by representatives from the implementing, 
supporting, using, and test commands. AFSPACECOM/LKY provides 
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support to the command managers at this meeting. R&M factors 
have a significant impact on system supportability so R&M 
progress should be monitored, problems discussed and cooperative 
action initiated at the ILSMT meetings. 


6.6 PREPARATION FOR TESTING 


Testing is a critical function for a system acquisition. 

In testing we evaluate a system throughout its acquisition cycle 
to insure the final product will meet system requirements. 
Preparing for this testing begins with the establishment of a 
Test and Evaluation Master Plan (TEMP). The test methods used 
to verify the performance parameters in Section 3 of the system 
specification are documented in Section 4 of the same 
specification. The action officer should ensure the test 
methods and parameters for verifying R&M requirements are also 
detailed in Section 4 of the specification before it is 
authenticated. Establishing a Joint R&M Evaluation Team (JRMET) 
for each program is also crucial to a successful R&M program. 


6.6.1 Test and Evaluation Master Plan (TEMP^ . The TEMP is 
drafted early in the conceptual phase by the SPO and the 
using/supporting/testing commands to outline the major elements 
of the test program (critical issues, test resources, timing and 
location). The TEMP critical issues are usually separated into 
two major themes: operational effectiveness and operational 
suitability. Operational effectiveness is concerned with how 
well the system operates in its intended environment-this is the 
operations part of the test. Operational suitability is 
bpncerned with how well the system can be supported and whether 
or not it is ready to perform its intended mission-this is the 
logistics support part of the test. R&M are critical parts of 
the operational suitability test objective and should evaluate if 
the system meets the operational values contained in the SORD's 
Requirements Correlation Matrix (RCM). The TEMP should be 
reviewed/coPjrdinated by AFSPACECOM/LKYY to insure it contains the 
proper R&M parameters. 


6.6.2 Joint R&M Evaluation Team (JRMET) . The JRMET is held 
regularly by the SPO'e.Systems Engineering Logistics Branch to 
review the TEMP, test plans and test data, establish proper data 
collection and data management procedures, correct deficiencies 
in data and, in general, ensure the R&M program is successfully 
completed. The JRMET will draft a charter outlining its 
functions and describing the responsibilities of each member. 



AFSPACECOM/LKY will normally attend these meetings as part of the 
logistics effort. The JRMET should also be attended by Command 
engineering personnel. The JRMET is a major avenue of addressing 
and raising issues on R&M during system acquisition, since all 
the key players (SPO, AFOTEC, AFLC, AFSPACECOM and the 
contractor) are in attendance. 









7.0 R&M IN THE TESTING PROCESS 

Section 6.6 discussed the importance of preparing for 
testing and the role of the JRMET. R&M testing is part of 
the two major test efforts. Development Test and Evaluation and 
Initial Operational Test and Evaluation continue throughout 
FSD,production deployment, and Operational Support Phases. The 
testing process reaches a high degree of detail in system/ 
subsystem testing, which includes the support elements of the 
system. The test and evaluation tasks are the primary means to 
verify achievement of program objectives and justify the 
continued or increased commitment of resources to the program. 

7.1 DEVELOPMENT TEST & EVALUATION (DT&E'l 

DT&E is the responsibility of the SPO. It tests system 
performance against system specifications to determine if the 
contractor has successfully implemented the required design. The 
DT&E test plan is written by the SPO and is supported by the 
contractor prepared/government approved DT&E test procedures. 
While the DT&E test environment is usually much more restrictive 
than the operational environment, the R&M data collected provides 
an initial data base of R&M experience. This data is passed to 
the JRMET for review, classification and analysis. An initial 
assessment of system R&M values is determined and corrective 
action initiated. 


7.2 OPERATIONAL TEST & EVALUATION (0T&E> 

OT&E provides the capability to track and evaluate the 
system's initial operational R&M performance; identify any 
deficiencies; and correct them through changes in design, 
technical data, software, support equipment, manning, training, 
or supply support. The OT&E process includes three primary 
areas: test planning, test execution and test reporting. 


7.2.1 Test Plans . OT&E test plans are written by AFOTEC (with 
assistance from the using/developing/supporting commands) if 
AFOTEC is conducting the test. AFSPACECOM will write the test 
plan if AFOTEC is monitoring the test. The test plans should 
show the test objectives, criteria and methodology that will be 
used to evaluate the R&M values outlined in the TEMP. The R&M 
values provide the core of the operational suitability 
evaluation, so it is important that adequate test planning be 
conducted prior to the test. Again, AFS PACE COM/LKYY can provide 
assistance in reviewing the test plans for consistency with the 
test objectives and in determining if the projected test data 
will provide adequate confidence in the test results. 
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7.2.2 Test Execution . Test execution is the actual test period 
during which the data is collected to make the final test 
report. The test is normally conducted using the contractors' 
test descriptions. Two of the tests are the reliability and 
maintainability demonstrations. The reliability demonstration is 
conducted to test how well the system performs without failure in 
the operational environment. The maintainability demonstration 
is conducted to evaluate the relative ease with which the design 
can be supported, verify maintainability requirements, and 
evaluate the effectiveness of fault detection/fault isolation. 
When the FSD objectives have been attained, the program reaches 
the production/acceptance decision. 


7.2.3 Test Reporting . The production/acceptance decision is 
supported by the OT&E test report. As in 7.2.1 above, whoever 
wrote the test plan (AFOTEC or AFSPACECOM) will write the test 
report. The test report details achieved R&M, explains any 
limitations that caused the test data to be less than what was 
desired, and identifies deficiencies in the achieved R&M versus 
the R&M standards described in the test plans. 




8.0 R&M IN THE OPERATIONAL ERA 

R&M tracking continues into the operational era with 
Follow-on OT&E (FOT&E) and maintenance data collection (MDC). 


8.1 FOLLOW-ON OPERATIONAL TEST & EVALUATION (FOT&E) 

FOT&E is conducted to test objectives not fully completed in 
OT&E, validate OT&E results and test corrections to deficiencies 
uncovered during OT&E. The production contract includes the 
requirement to prevent previously demonstrated levels of R&M 
performance from being degraded and ensure that R&M is verified 
periodically during the production run. Successful R&M 
demonstration is a condition of acceptance of production 
articles. Some programs may contain Reliability Improvement 
Warranties (RIWs) to incentivize production contractors to 
improve system reliability. By correcting design problems or 
defects that reduce system reliability, the contractor can reduce 
costs incurred by contract warranty clauses. 


8.2 MAINTENANCE DATA COLLECTION CMPCM 

MDC is the system whereby the maintenance downtime is 
tracked and analyzed so that actual R&M performance can be 
determined. AFSPACECOM/LKM is the OPR for MDC. MDC should be 
performed even though the system is contractor maintained. This 
allows AFS PACECOM to evaluate contractor as well as system 
performance. This data is used to determine trends in 
maintenance and project when the system needs to be replaced or 
modified. It's important during this phase to ensure that 
standardized (both in form and method of delivery) MDC 
requirements are placed on all contracts for maintenance and 
logistics support in the operational era. The Air Force Standard 
MDC system is the Core Automated Maintenance System (CAMS). 



9.0 SUMMARY 


Establishing realistic and consistent R&M parameters in SONs 
and SORDs is critical to fielding a system that will meet our 
operational requirements and be logistically supportable. Your 
responsibility for R&M does not stop with these requirements 
documents. Translation of operational and logistics support 
requirements into contractual clauses is necessary. This 
translation is a primary responsibility of the System Program 
Office; the using command is responsible for assisting in this 
effort. 

The R&M requirements and deliverables specified in the 
system specifications, statement of work, and contract data 
requirements list must be closely examined by the using command. 
Tracking R&M allocation through specification and design reviews 
is especially important. 

Testing R&M parameters is an integral part of DT&E and IOT&E 
efforts. DT&E basically tests contract R&M parameters; IOT&E 
tests operational R&M parameters. 

Continued emphasis on and attention to R&M are required 
throughout the life cycle of the system. Once the system is 
fielded, R&M data must be collected through a standard 
maintenance data collection system and subsequently analyzed to 
identify any R&M problems. Corrective action up to and including 
system replacement will be evaluated. This analysis may even be 
used to justify the preparation of a new SON/SORD. 
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APPENDIX A 
R&M PRIMER 



RELIABILITY AND MAINTAINABILITY 
PRIMER 


1.0 INTRODUCTION 

The purpose of this primer is to provide the action officer 
an overview of the concepts, terminology and relationships used 
in Reliability and Maintainability (R&M). It is not designed to 
make the action officer an R&M engineer. The terms and 
definitions used are taken from MIL-STD-721C, AFSPACECOM/LKY 
Technical Report 88-1, and various R&M texts. 

This primer starts with an explanation of basic R&M concepts. 
Next these basic concepts are expanded and refined to include 
mission and logistics R&M parameters. 


2.0 BASIC CONCEPTS 

This section defines the basic concepts of reliability and 
maintainability, introduces item level R&M terms, and shows how 
to do rudimentary R&M calculations. These terms are not to be 
used to state system level operational R&M requirements. 

2.1 Reliability Reliability is the duration or probability of 
failure free performance under stated conditions. It can 

also be defined as the probability an item can perform its 
intended functions for a specified interval under stated 
conditions. 

2.1.1 Mean Time Between Failures (MTBF) A basic quantitative 
measure of reliability is Mean Time Between Failures (MTBF). 

This is the expected length of time a system will be operational 
between failures. It is normally calculated by taking the number 
of operational hours in a stated period and dividing it by the 
number of failures. MTBF could be expressed in other life units 
such as number of cycles, number of orbits, or number of 
landings. 

2.1.2 Failure Rate Failure rate is defined as the number of 
failures of an item per measure of life units (e.g., failures per 
million hours, failures per 1000 cycles). The failure rate is 
simply the reciprocal of the MTBF. If the average time between 
failures is 1,000,000 hours (i.e., MTBF = 1,000,000), then the 
failure rate is 1 failure per 1,000,000 hours or 0.000001 
failures per hour. In some computations failure rates are 
simpler to use than the associated mean (average) value. 

2.1.3 Mission Reliability (MR! The second definition of 
reliability stated in paragraph 2.1 includes the added factor of 
operating "for a specified interval". Instead of determining the 
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average time between failures or the number of failures per hour, 
Mission Reliability (MR) deals with the probability of the item 
working continuously without a failure for a specified period of 
time. The term most commonly used for the specified time period 
is Mean Mission Duration. The second factor affecting MR is 
MTBF. The mathematical equation for MR for most electronic items 
is: 

-MMD/MTBF 

* MR ■ e 

Where: e = base of natural logarithms = 2.71828 

MTBF = Mission Time Between Failures 
MMD = Mean Mission Duration 

This is the most difficult mathematical equation we will use in 
this primer; most hand held scientific or business calculators 
will easily handle this equation. An interesting note about MR: 
if you design a system with a MTBF equal to the length of the 
average mission (i.e., MTBF = MMD), the probability of surviving 
that mission is only 36.8%. Mission Reliability will be further 
\ discussed in Section 3.6. 

\ 2.2 Maintainability is defined as a characteristic of a design 

\that determines the type and amount of maintenance required to 
retain that design in, or restore it to, a specified condition. 
Maintenance required to retain an item in its designed condition 
is normally called Preventive Maintenance (PM) since it prevents 
a failure from occurring. Maintenance required to restore an 
item isxnormally called Corrective Maintenance (CM) since it 
corrects^some fault in the system. 

2.2.1 Mean Time To Repair (MTTR) The basic measure of item 
maintainability is Mean Time To Repair (MTTR). MTTR is 
calculated by dividing the number of times an item required 
jrepair into the total length of time required to make those 
repairs. MTTR includes active maintenance time only. It does 
not include any delay time (e.g., time spent waiting for parts). 

2.2.2 Administrative and Logistics Delay Time (ALDT) There are 
factors external to the item design that affect the actual time 
taken to perform maintenance. Some of these factors are the time 
required for the technician to arrive at the failed item, time 
spent waiting for tools or equipment to test the item, and time 
spent waiting for parts. The delays caused by these external 
factors are generically called Administrative and Logistics Delay 
Time (ALDT). 

2.2.3 Mean Down Time (MPT) It is often important to know the 
time required to restore a failed item including expected delays. 
Mean Down Time (MDT) is the term used for this. MDT is the sum 
of the active maintenance time plus administrative and logistics 
delays (i.e., MDT = MTTR + ALDT). 
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2.3 Availability The probability an item is available for use 
is a function of how often it breaks and how long it takes to 
repair. The formal definition of availability is "a measure of 
the degree to which an item is in an operable and committable 
state at the start of a mission when the mission is called for at 
any random time. Availability is sometimes referred to as the 
up time ratio. Simplistically it is Uptime/Total Time or 
Uptime/(Uptime + Downtime). Uptime is a measure of the item's 
reliability and downtime a measure of its maintainability. 

2.3.1 R&M Trade Offs There are many combinations of R&M 
values which could yield the same Availability value. For 
example an item that is up (operational) 100 hours and down 
(failed) 1 hour has the same availability as an item up 400 hours 
and down 4 hours [100/(100+1) = 400/(400+4)]. It may be 
unacceptable for the item to fail more frequently than every 200 
hours or to take more than four hours to repair. For this reason 
it's necessary to specify more than just the item's availability. 
Figure 2-1 shows graphically the interrelationship among 
reliability, maintainability and availability. 



2.3.2 Inherent Availability (Ai) Availability can refer to the 
inherent capability of the item, independent of the support 
infrastructure. This type of availability is referred to as 
inherent availability (Ai). The measure used for Uptime is 
normally MTBF. The measure used for Downtime is MTTR. The 
resultant formula is Ai = MTBF/(MTBF + MTTR). 
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2.3.3 Operational Availability (Ao) To determine the 
availability expected in the normal operating environment, it 
becomes necessary to account for the impact of support delays. 
This type of availability is called operational availability 
(Ao). The factor used for Uptime if we assume there is no 
interfering preventive maintenance, is the same one used in Ai 
(i.e., MTBF). The Downtime factor used is Mean Down Time (MDT) 
which equals MTTR + ALDT. The formula is Ao = MTBF/(MTBF + MDT). 

2.4 Series/Parallel Models The previous sections introduced 
the basic R&M concepts as applied to individual items. Items are 
normally combined to perform more complex functions. The way 
they are combined affects the reliability of that combination. 
Items can be combined in series or in parallel. 

2.4.1 Series Reliability When units are combined in such a way 
that a function must be successfully performed by the first 
unit before the second unit can perform its function and so on, 
the units are said to be combined in a series configuration. 
Figure 2-2 is a typical series reliability block diagram. 


SERIES RELIABILITY 



R = 0.90 R = 0.90 R = 0.90 


Figure 2-2 


In figure 2-2, a function requires items A, B, and C to operate. 
Assume each block has a probability of operating of 0.90. To 
find the probability of completing that function, we must 
consider the probability of A, B, and C each working. 
Mathematically we do this by multiplying the reliability of each 
item (i.e., Ra, Rb, & Rc). The formula for this set of three 
components is "Rs = Ra x Rb x Rc. In this case the resultant 
reliability is only 72.9% (i.e.,0.9x0.9x0.9). 

As this example indicates, even though individual items may have 
good reliability, their serial combination results in a 
significantly lowered reliability. 

2.4.2 Parallel Reliability There is a way to combine items so 
that the combined reliability is better than the individual 
reliabilities. When items are tied together so their function 
can be performed by any one (or more) of them, they are said to 
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In Figure 2-3, as indicated by the 1/3, only one of the three 
items must be working for the function to be performed. In other 
words the function fails only when all three items fail 
simultaneously. We can find the probability of this happening by 
multiplying the individual probabilities of failing. 

Let's assume items D,E, and F each have a probability of not 
failing (Reliability) of 0.90. Since the probability of failing 
added to the probability of not failing must equal one, the 
probability of failing equals 0.10, (i.e., 1 - 0.9). The chance 

of all three items failing is 0.1 x 0.1 x 0.1 = .001. Since we 
now know the probability of all three failing, we subtract it 
from one to get the probability of one or more not failing (1.0 - 

0.001 = 0.999). So in this configuration, the probability of the 

function being performed is 0.999. 

Putting this logic into a formula we get: 

If reliability of D is Rd then its unreliability is 1 - Rd 

If reliability of E is Re then its unreliability is 1 - Re. 

If reliability of F is Rf then its unreliability is 1 - Rf. 

The probability of D, E, & F failing is (1-Rd)(1-Re)(1-Rf). 
The probability of D, E, & F not failing (Rs) is 

Rs - 1 -[(1-Rd)(1-Re)(1-Rf)] 

It's also possible to put items in parallel and require more than 
one to remain operational. Computing the probability of the 
function being performed requires the use of complex mathematical 
computations outside the scope of this primer. 

2.4.3 Series/Parallel Combinations In more complex 
configurations there are often combinations of series and 
parallel items. In these cases it is relatively simple to reduce 
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3.0 MISSION R&M 

The concepts introduced in section two are basic, 
component level concepts. These concepts can be applied to 
system level requirements with some minor modifications and _ 
expansions. This section will define the R&M terms for use in 
defining operational requirements in Statements of Operational 

Need (SONS) and System Operational Requirements Documents 

(SORDs). These terms are especially adaptable for use with 
space, space surveillance and missile warning systems. 

3.1 Mission Effectiveness (ME)_ System capability has two 
factors associated with it. The first factor is the probability 
of a system being operable and capabie of initiating a missio: n _at 
any (random) time. The second factor is the probability that 
system is operable and capable at any (random) timeduring a _ 
mission. If these two probabilities are expressed in consisten 
terms, then the probability of effectively completing the mission 
is the product of the two factors. 

The first factor is the traditional definition of system _ 
availability. The second factor is a measure of how dependable a 
system is once a mission has begun. In this context, 
availability is associated with non-mission time and 
dependability with mission time. Since the system ^frequently 
inactive or exercised differently during non T mission time, there 
are different non-mission and mission reliability and . 

maintainability factors associated with it. Figure 3-1 shows the 
relationships among these factors. 

SYSTEM TIME VERSUS R&M PARAMETERS 


MISSION EFFECTIVENESS 
(TOTAL TIME) 


AVAILABILITY 
(NON-MISSION TIME) 


INACTIVE TIME ACTIVE TIME 


DEPENDABILITY 
(MISSION TIME) 

ACTIVE TIME 


Figure 3-1 


3 2 Availability & Dependability The terms availability and 
dependability are not new. These terms are listed in 
MiL-STD-721C, Definition of Terms for Reliability and 
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Maintainability. Although the term availability has always been 
defined as the probability of being operable and capable at any 
(random) time of initiating a mission, it has often been used to 
calculate the probability of the system working at any (random) 
time during a mission. 

The terms and concepts of availability and dependability are 
also well documented in standard logistics engineering and 
reliability texts. Examples are Benjamin S. Blanchard's 2nd 
edition of Logistics Engineering and Management . Page 20, and 
Igor Bazovsky's Reliability Theory and Practice . Chapter 17. 

Misuse of the term availability hasn't created many problems in 
the aircraft world because frequently only mission time is 
involved. However, if we are to effectively account for the 
unique R&M parameters associated with spacecraft launch, on orfoi 
mission, and space vehicle turn around, a return to the more 
accurate definitions is required. 

3*3 Mission Time R&M Parameters The R&M parameters associated 
with non-mission time can be inherently different from those with 
mission time (e.g., Age deterioration of seals versus wearout 
failures). MIL-STD-721C defines the reliability and 
maintainability parameters associated with dependability (mission 
time) as Mission Time Between Critical Failures (MTBCF) and 
Mission Time To Restore Functions (MTTRF). As their titles 
indicate, they are only concerned with the impact on mission. 

This makes them ideal for stating operational requirements and 
for documenting results of operational testing. 

3.3.1 Mission Time Between Critical Failures (MTBCF) The 
definition of Mission Time Between Critical Failures from 
MIL-STD-721C is: A measure of mission reliability. The total 
amount of mission time, divided by the total number of critical 
failures during a stated series of missions. Its formula is: 

TOTAL MISSION TIME 

MTBCF ---- 

# OF CRITICAL FAILURES 

3.3.2 Mission Time To Restore Functions fMTTRF^ The definition 
of Mission Time To Restore Functions is: A measure of mission 
maintainability. The total maintenance time required to restore 
mission functions interrupted by critical failures divided by the 
number of critical failures during the course of a specified 
mission profile. MTTRF includes both scheduled and unscheduled 
maintenance. Its formula is: 

TOTAL RESTORAL TIME 

MTTRF =----—- 

# OF CRITICAL FAILURES 

When MTTRF is used to calculate Operational Dependability (Do) 
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versus Inherent Dependability (Di), Administrative and Logistics 
Delay Time (ALDT) is included as part of total restoral time. 

3.3.3 Maximum Mission Time To Restore Functions _(MaxTTRF) If 

only the mean MTTRF value is specified, there may be a wide range 
of individual repair time values. Figure 3-2 shows two repair 
time distributions with the same mean value. The lower curve has 
a greater variation around the mean value. Two systems could be 
designed to meet the same mean MTTRF specification and not 
necessarily meet operational requirements. A system design which 
follows the lower distribution will have more failures that take 
longer to repair than one whose design fits the upper curve. 

This could result in repairs taking longer than is operationally 
acceptable, on too frequent a basis. 


REPAIR TIME DISTRIBUTIONS WITH THE SAME MEAN 
BUT WITH DIFFERENT VARIANCE 

P(t) 



Because of this, it may become necessary to place restrictions on 
the maximum amount of time the system must be restored within. 
This can be done by specifying a Maximum Time To Restore 
Functions (MaxTTRF) at a stated percentile. For example, you 
could specify that 90% of all repairs be accomplished in 2 hours 
or less. This parameter is sometimes referred to as Maximum 
Continuous Downtime. 

3.3.4 MaxTTRF and Availabilitv/Dependability As seen earlier in 
this primer, availability and dependability are normally based on 
mean values. If we know the maximum time we want the system 
repaired in, we need a way to find the mean value for use in 
these calculations. There is a relationship between the mean and 
maximum values for a given distribution. By knowing what the 
underlying distribution is, we can calculate the mean value. For 
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more information on how to convert mean and max values, see 
Section 3.5 on distributions. 

3.4 Non-mission Time R&M Parameters During non-mission time it 
is possible to have active and inactive time. There can be 
inherent characteristics of the system that affect R&M parameters 
during these times. Under certain circumstances, it may be 
logical to separate the R&M factors associated with inactive 
non-mission and active non-mission time. However, under normal 
circumstances it usually is adequate to address non-mission time 
R&M factors as a single entity. 

3.4.1 Mean Time Between Downing Events (MTBDE) A downing event 
is any event during non-mission time that affects the system's 
ability to initiate a mission. Scheduled interfering preventive 
maintenance or servicing actions required to maintain the system 
in a state capable of initiating a mission are examples of 
downing events. MIL-STD-721C calls MTBDE the reliability 
parameter associated with readiness. 

The definition of MTBDE is: A measure of system reliability 
associated with availability. The total non-mission time divided 
by the # of downing events. It formula is: 

TOTAL NON-MISSION TIME 

MTBDE = —- 

# OF DOWNING EVENTS 

3.4.2 Mean Time To Restore System (mttrs) The collateral 
maintainability criteria associated with readiness is Mean Time 
To Restore System (MTTRS). MTTRS applies only to maintenance 
actions that occur during non-mission time. MTTRS includes both 
scheduled and unscheduled maintenance actions. 

The definition of MTTRS is: A measure of system maintainability 
associated with availability. The total maintenance time 
associated with downing events divided by the number of downing 
events. Its formula is: 

Total Restoral Time 

MTTRS = - 

# of Downing Events 

When MTTRS is used to calculate Operational Readiness (Ro) versus 
Inherent Readiness (Ri), total restoral time includes 
Administrative and Logistics Delay Time (ALDT). 

3.5 Distributions Failure rates and repair times tend to follow 
a general pattern for specific types of systems. The general 
pattern followed fits a specific statistical distribution. 

3.5.1 Failure Rate Distributions There are three basic types of 
failures for communication-electronic systems. These are burn-in 
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failures, random failures and wear out failures. Burn-in 
failures are normally screened out by testing or environmental 
stress screening prior to placing the system in field operation. 
During the burn-in period, the failure rate normally decreases; 
random failures occur during the useful life period. The failure 
rate during the useful life period remains relatively constant. 
During the wearout period the failure rate increases. These 
three types of failure rates are shown graphically in Figure 3-3. 
This particular curve is commonly known as the bathtub curve. 


COMPONENT FAILURE RATES AS A FUNCTION OF TIME 



Figure 3-3 


Since burn-in failures normally occur during the development/ 
production phase, we are only concerned about random and wearout 
failures in this primer. 

3.5.1.1 Exponential Failure Distribution Failures during the 
useful life period for communication-electronic components follow 
an exponential distribution. With this type of distribution, 

63.2 percent of the failures will occur before the mean value. 
From another perspective the mean value will be met or exceeded 
only 36.8 percent of the time. Figure 3-4 is a graph of the 
exponential probability density function. 

3.5.1.2 Normal Failure Distribution Component failures caused 
by wearout follow a normal distribution. Figure 3—5, shows the 
normal wearout of a lamp. The average life expectancy for this 
particular lamp is 7200 hours. A constant level of random 
failures would be expected during the useful life period. Once 
the useful life period is over (around 5000 hours), the lamps 
will start experiencing wearout failures. By 7200 hours we would 
expect half the lamps to have failed. The other half would fail 
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after 7200 hours. The rate of failure would be symmetrical 
around the mean value. 










3.5.2 Restoral Time Distributions Like failure rates, restoral 
times also tend to follow specific patterns or distributions. For 
communication-electronic systems three types of distributions are 
commonly found. The type of maintenance actions required to 
restore the system affects which of the three is appropriate. 


3.5.2.1 Exponential Repair Distributions When system restoral is 
primarily accomplished by a combination of manual and automatic 
switchover to a redundant unit, an exponential distribution is 


normally followed. This curve is shown in Figure 3-6. As seen 
from this graph, the majority of items can be repaired within a 
short restoral time with fewer restoral actions occurring as 


repair time increases. As with the exponential failure rate 
curve, 62.8 percent of the repair actions will be accomplished 
before the mean repair time is reached. 



Mean (MDT) and maximum (Mmax) allowable downtimes are shown in 
Figure 3-6. The 0.69 sigma indicates the number of standard 
deviations the mean is away from the origin (time zero). The 2.30 
sigma is the number of standard deviations the 90th percentile is 
away from the origin. The hatched area indicates repair times 
that are equal to or less than the maximum allowable down time. 

From this information we can establish a ratio between the mean 
and maximum values. For instance, if we want 90 out of 100 
repair actions to be accomplished in 1 hour or less, a mean value 
of 0.30 hours [(0.69/2.30) (1 hour)] is expected. Similarly, if 
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we specify a mean value of 1 hour, 90 out of 100 repair actions 
will be accomplished within 3.33 hours [(2.30/0.69) (1 hour)]. 

3.5.2.2 Log-Normal Repair Distributions When system restoral is 
accomplished primarily by a combination of switchover and 
on-equipment repair, a Log-Normal distribution is often seen. 
Figure 3-7 depicts a typical Log-Normal probability density 
function with mean (MDT) and maximum (Mmax) allowable downtime 
shown. The 0.8 sigma indicates the number of standard deviations 
the mean is away from the origin (time zero). The 1.86 sigma is 
the number of standard deviations the 90th percentile is away 
from the origin. The hatched area indicates repair times that 
are equal to or less than the maximum allowable down time. 



From this information we can establish a ratio between the mean 
and maximum values. For instance, if we want 90 out of 100 
repair actions to be accomplished in 1 hour or less, a mean value 
of 0.43 hours [(0.80/1.86) (1 hour)] is expected. Similarly, if 

we specify a mean value of 1 hour, 90 out of 100 repair actions 
will be accomplished within 2.325 hours [(1.86/0.80) (1 hour)]. 

3.5.2.3 Normal Repair Time Distribution When system restoral is 
accomplished primarily by on-equipment repair actions with 
associated administrative and logistics delays, a normal 
distribution is usually found. As with the normal failure rate 
distribution, 50 percent of the repair actions will be 
accomplished before the mean value and 50 percent after. The 
normal probability density function including mean (MDT) and 
maximum (Mmax) allowable downtime for repair actions is shown in 
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Figure 3-8. The 3.0 sigma indicates the number of standard 
deviations the mean is away from the origin (time zero). The 
4.28 sigma is the number of standard deviations the 90th 
percentile is away from the origin. The hatched area indicates 
repair times that are egual to or less than the maximum allowable 
down time. As with the exponential and log-normal distributions, 
we can establish a ratio between the mean and maximum values. For 
instance, if we want 90 out of 100 repair actions to be 
accomplished in 1 hour or less, a mean value of 0.70 hours 
[(3.00/4.28) (1 hour)] is expected. Similarly, if we specify a 

mean value of 1 hour, 90 out of 100 repair actions will be 
accomplished within 1.43 hours [(4.28/3.00) (1 hour)]. 



3.5.3 Mean/Maximum Conversions If operational considerations 
require a specified maximum allowable downtime, the above 
transformations become especially useful. It is possible to 
specify percentile values other than the 90th percentile but 
usually not feasible to design a system so that 100% of all 
possible failures can be repaired in a stated period. The 90th 
percentile is normally used because it represents an efficient 
tradeoff between cost and repair time. 

AFOTEC has collected and analyzed repair time data from a 
multitude of communication-electronic systems. After applying 
curve fitting techniques, they have found that the log-normal 
distribution typifies these systems. With this assumption we can 
make a reasonable approximation of the mean value associated with 
specific maximum values. 
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.3.6 Mission Reliability (MR) 

The concept of mission reliability that was discussed in the 
basic concepts section can be applied directly at the system 
level. Instead of MTBF, the term Mission Time Between Critical 
Failures (MTBCF) is used. Mission reliability is a measure of 
system reliability during mission time. It does not take into 
account system maintainability or availability factors. 

There are many missions and systems which do not allow restoral 
of specific functions during the mission. Consider the example 
of a spacecraft oxygen or propulsion system. If a critical 
failure occurs, it may not be possible to restore the system 
prior to running out of oxygen or in time to achieve orbit. It 
then becomes an operational requirement to achieve a stated 
reliability for a stated mission duration. 

The definition of Mission Reliability (MR) is: A measure of the 
degree to which a system is operable and capable of performing 
its required functions at a specified time into a mission or for 
a mission of stated duration. It is based on the effects of 
mission reliability during mission time only. 

The concept of mission reliability is based on the mathematical 
probability of survival function. The formula for mission 
survivability will vary based on the underlying distribution of 
mission critical failures. For an exponential distribution the 
formula for MR is: 

-MMD/MTBCF 

MR = e 

Where: e = base of natural logarithms 

MTBCF = Mission Time Between Critical Failures 
MMD = Mean Mission Duration 


3.7 Mission Versus Logistics Parameters The R&M parameters 
discussed above deal with the effects on mission accomplishment. 
They are of prime importance to the operational commands. The 
operational commands are also very concerned with a second set of 
R&M parameters that deal with logistics support related terms. 
These parameters have significant impact on command manpower and 
supply support. Section 4 of this primer addresses these 
factors. 
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4.0 LOGISTICS R&M 

Although mission R&M parameters are an indication of the 
capability of the system to perform specified mission profiles, 
they are not the only ones operational commands are concerned 
with. This section will address the R&M parameters associated 
with logistics support and their impact on the operational 
command. 

4 .1 Maintenance Concerns The operational command is concerned 
with R&M factors that affect maintenance. Of special interest 
are the areas of maintenance manpower and maintenance cost. These 
parameters may be used in requirements documents when they 
specify operational constraints the system must be designed 
within. An example is a system that must be designed to be 
maintained with the same or fewer number of personnel as the 
system it replaces due to manpower ceiling limitations. Another 
example is a contractor maintained system whose Operations and 
Maintenance (O&M) budget must not exceed programmed contract 
maintenance funds. 

4.2 Mean Time Between Maintenance (MTBM) Regardless of the 
impact on mission effectiveness, every maintenance action can 
have an impact on maintenance manning. The number of maintenance 
actions and the length of time to perform the average maintenance 
action combine to determine the basic manpower requirement. The 
reliability term associated with maintenance manning is Mean Time 
Between Maintenance (MTBM). 

Since the purpose of this term is to determine the impact on 
maintenance manpower and cost, all maintenance actions should be 
considered. This includes mission/non-mission time as well as 
scheduled/unscheduled maintenance actions. 

The definition of MTBM is: A measure of system reliability 
taking into account maintenance policy. The total number of 
system life units in a stated period of time divided by the 
number of maintenance events both scheduled and unscheduled. The 
formula is: 



Since the frequency of MTBSM and MTBUM may vary, it is necessary 
to convert then to rates, add them, and take the reciprocal. 
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4.2.1 Mean Time Between Scheduled Maintenance (MTBSM1 Mean Time 
Between Scheduled Maintenance is the term used to indicate the 
average frequency of scheduled maintenance events. The term 
'•preventive maintenance" is frequently used to denote these types 
of maintenance events. Preventive maintenance has the 
connotation of actions taken specifically to prevent a failure. 

On certain space and missile warning systems, routine servicing 
activities (e.g., refueling of a satellite) may occur. In some 
respects these servicing actions are not preventive maintenance 
actions. For this reason, the use of the more generic term 
scheduled maintenance is proposed. 


The definition of Mean Time Between Scheduled Maintenance (MTBSM) 
is: A measure of system reliability taking into account 
maintenance policy. The total relevant system time divided by 
the number of scheduled maintenance events. The formula for 
MTBSM is: 


MTBSM - 


TOTAL SYSTEM TIME 
# OF SCHEDULED MAINTENANCE EVENTS 


4.2.2 Mean Time Between Unscheduled Maintenance (MTBUM1 Mean 
Time Between Unscheduled Maintenance is the term used to denote 
the average frequency of unscheduled maintenance events. The 
term unscheduled maintenance is proposed instead of corrective 
maintenance for two reasons. The first reason is that certain 
preventive maintenance events (e.g., repainting for corrosion 
control) are done on an as required (i.e., unscheduled) basis. 
The second reason is to be consistent with the term scheduled 
maintenance. 


The definition of Mean Time Between Unscheduled Maintenance 
(MTBUM) is " A measure of system reliability taking into account 
maintenance policy." The total relevant system time divided by 
the number of unscheduled maintenance events. The formula for 
MTBUM is: 


MTBUM = 


TOTAL SYSTEM TIME 

# OF UNSCHEDULED MAINTENANCE EVENTS 


4.3 Mean Maintenance Time (MMT) While MTBM, MTBSM, and MTBUM 
are measures of system reliability, each have a collateral 
measure of system maintainability. The collateral 
maintainability term for MTBM is Mean Maintenance Time (MMT). 

The definition of MMT is: A measure of system maintainability 
considering maintenance policy. The sum of unscheduled and 
scheduled maintenance times divided by the sum of scheduled and 
unscheduled maintenance events during a stated period of time. 
The formula is: 
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SCHEDULED + UNSCHEDULED MAINTENANCE TIME 

MMT --*--- 

# OF SCHEDULED + UNSCHEDULED MAINTENANCE EVENTS 

4.3.1 Mean Scheduled Maintenance Time (MSMT) The 
maintainability parameter associated with Mean Time Between 
Scheduled Maintenance (MTBSM) is Mean Scheduled Maintenance Time 
(MSMT). The definition of MSMT is: A measure of system 
maintainability taking into account maintenance policy. Total 
scheduled maintenance time divided by the number of scheduled 
maintenance events during a stated period of time. The formula 
is: 

TOTAL SCHEDULED MAINTENANCE TIME 

MSMT = -- 

# OF SCHEDULED MAINTENANCE EVENTS 

4.3.2 Mean Unscheduled Maintenance Time (MUMT) The 
maintainability parameter associated with Mean Time Between 
Unscheduled Maintenance (MTBUM) is Mean Unscheduled Maintenance 
Time (MUMT). The definition of MUMT is: A measure of system 
maintainability considering maintenance policy. Total 
unscheduled maintenance time divided by the number of unscheduled 
maintenance events. The formula is: 

TOTAL UNSCHEDULED MAINTENANCE TIME 

MUMT = - 

# OF UNSCHEDULED MAINTENANCE EVENTS 

4.4 Supply Concerns The operational command is also concerned 
with R&M parameters that affect supply support. The frequency of 
demands on the supply system and the cost of that support are two 
areas of special interest. Although supply R&M parameters are 
normally used to assess support cost of a specified design, they 
can also be used to state operational constraints. Mission 
deployment requirements may limit the demands that can be placed 
on supply. Budget limitations may require the supply support 
costs to be within a programmed amount (e.g., Supply support 
contract dollars are limited). 

4.4.1 Mean Time Between Demand (MTBD) Regardless of the impact 
. on mission effectiveness, every demand on supply affects supply 
support cost. The cost of this support must be programmed, 
planned and budgeted. The average frequency of demands and the 
average cost of each demand are used to assist in determining the 
amount to be funded. The measure of the reliability related to 
the frequency of demands placed on supply is Mean Time Between 
Demand (MTBD). The definition of MTBD is: A measure of the 
system reliability parameter related to supply support. The 
total number of system life units divided by the total number of 
items demanded from supply during a stated period of time. The 
formula for MTBD is: 
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MTBD 


TOTAL LIFE UNITS 


# OF ITEMS DEMANDED 


4.4.2 Mean Parts Cost/Demand (MPC/D) Once the frequency of 
demands and the average cost of each demand is known, supply 
support costs can be estimated. The term associated with the 
cost of demands is Mean Parts Cost/Demand (MPC/D). The 
definition of MPC/D is: A measure of system support costs 
related to supply reliability. The total cost of parts demanded 
from supply divided by the number of demands during a stated 
period of time. The formula is: 

TOTAL PARTS COSTS 

MPC/D ---- 

# OF DEMANDS 
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APPENDIX B 


SYSTEM OPERATIONAL REQUIREMENTS DOCUMENT CHECKLIST 



SQRD CHECKLIST 


s 


1. Does the SORD address, as a minimum, these Reliability 

and Maintainability factors: 

a. What is the required system R&M effectiveness? 

b. Based on system's employment and deployment concepts 
what is the system's required availability? 

c. What is the required dependability once a mission is 
initiated? 

d. What is required mission reliability for a mission of 
a stated duration? 

e. ; What is the maximum acceptable system restoral time? 

f. What frequency of critical software failures is 
acceptable? 

g. What frequency of hardware failures is acceptable? 

h. What maintenance and operations manpower constraints 
will the system be required to be operated in. 


2. Do the above parameters reflect improved R&M system 

| performance parameters over the system(s) being replaced? 

' 3 . Concerning maintenance, does the SORD address 

a. Levels of maintenance? 

b. Skill level of blue-suit, robotic or contractor 
^personnel designated to maintain the system? 

'■ * J , . * t 

4. Does? the SORD specify^quantitative values for operational 
and logistics support performance parameters? 

5. Do quantitative and qualitative readiness requirements 
reflect the command R&M goals? 

• .i- 

6. Are the R&M terms integrated such that: 

a. R&M parameters are explained in terms of how they 
affect operational capability? 

b. R&M terms used in the SORD are defined sufficiently 

for conversion into contractual terms in a future 
System Specification? V 



7. 


c. R&M goals and requirements are reasonable and 
compatible? 

Are values for built-in test equipment (BITE) 
effectiveness defined? 

8. Is the level of diagnostics defined? 

Has software R&M been addressed? 


9. 




APPENDIX C 


R&M DATA ITEM DESCRIPTION LIST 



RELIABILITY DATA ITEM DESCRIPTIONS (DID) 

The following is a list of data item descriptions associated with 
the reliability tasks specified in MIL-STD-785B. 


TASK 

APPLICABLE DID 

DATA REQUIREMENT 

101 

DI-R-7079 

Reliability Program Plan 

103 

DI-R-7 080 

Reliability Status Report 

104 

DI-RELI-80255 

Report, Failure Summary and Analysis 

201 

DI-R-7081 

Reliability Mathematical Models 

202 

DI-R-2114 

Report, Reliability Allocation 

203 

DI-R-7082 

Reliability Predictions Report 

204 

DI-R-7 085A 

Report, Failure Modes, Effects and 
Criticality Analysis Report (FMECA) 


DI-R-7 086 

FMECA Plan 

205 

DI-R-7083 

Sneak Circuit Analysis Report 

206 

DI-R-7 084 

Electronic Parts/Circuits Tolerance 
Analysis Report 

208 

DI-R-30511 

Plan, Critical Item Control 

The following tasks have DIDs associated with them related to 
imposition of MIL-STD-781C: 

301 

DI-R-7040 

Report, Burn-in Test 

302, 

303, 
304 

DI-RELI-80250 

Plan, Reliability Test 

303, 

304 

DI-RELI-80251 

Procedures, Reliability Test and 
Demonstration 
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303, 

304 DI-RELI-80252 Reports, Reliability Test and 

Demonstration (Final report) 


Notes: (1) Only data items specified in the CDRL are 

deliverable. Therefore, those data requirements identified in 
the Reliability Program Plan must also appear in the CDRL. 

(2) The PA should review all DIDs and assure, through * 

tailoring, that the preparation instructions in the DID are 
compatible with task requirements as specified in the SOW. 


MAINTAINABILITY DATA ITEM DESCRIPTIONS (DID) 


The following is a list of data item descriptions associated with 
the maintainability tasks specified in MIL-STD-470A. 


101 

DI-R-7103 

Maintainability Program Plan 

103 

DI-R-7104 

Maintainability Status Report 

104 

DI-R-7105 

Data Collection, Analysis and 
Corrective Action System, Reports 

201 

DI-R-7106 

Maintainability Modeling Report 

202 

DI-R-7107 

Maintainability Allocations Report 

203 

DI-R-7108 

Maintainability Predictions Report 

204 

DI-R-7085A 

Failure Mode, Effects and Criticality 
Analysis (FMECA) Report 

205 

DI-R-7109 

Maintainability Analysis Report 

206 

DI-R-7110 

Maintainability Design Criteria Plan 

207 

DI-R 7111 

Inputs to the Detailed Maintenance 
Plan and Logistics Support Analysis 

301 

DI-R-7112 

Maintainability Demonstration Test 
Plan 
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301 DI-R-2129 


301 DI-R-7113 


Plan, Maintainability Demonstration 
(DI -R-2129 is to be used only when 
MIL-STD-471 is designated as the basis 
for MIL-STD-470A, Task 301) 

Report, Maintainability Demonstration 
(to be used only when MIL-STD-471 and 
its associated DI-R-2130A are not 
designated as a basis for 
MIL-STD-470A, Task 301) 


Notes: (1) Only data items specified in the CDRL are 

deliverable. Therefore, those data requirements identified in 
the Reliability Program Plan must also appear in the CDRL. 

(2) The PA should review all DIDs and assure, through 
tailoring, that the preparation instructions in the DID are 
compatible with task requirements as specified in the SOW. 
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The following Data Item Descriptions are listed for reference 
purposes only. They have not been linked to any specific task 
number in any MIL-STD. 


Data Item 


Description 


DI-RELI-80247 
DI-RELI-80248 
DI-RELI-80249 
DI-RELI 80253 
DI-RELI-80254 
DI-RELI-80261 

DI-RELI-80322 

DI-RELI-80323 

DI-RELI-80373 

DI-RELI-80374 

DI-3549A 

DI-7094 

DI-R-3541/R-109-2 

DI-R-3547/R-115-2 


Thermal Survey Report 
Vibration Survey Report 
Environmental Stress Screening Report 
Failed Item Analysis Report 
Corrective Action Plan 

Production Inspection Equipment Test Systems 
Engineering Design Data 

Quality Conformance Inspection & Test 
Procedures 

Certification Demonstration Procedures 

Equipment Inspection/Testing Report 

Failure ANalysis Summary List 

Repair Level Analysis Report (RLA) 

Reliability Block Diagrams and Math 
Models Report 

Computer-Programmed Math Model for 
Reliability 

R&M Report on Commercial Equipment 



